1. Field of the Invention
The present invention relates to a subject tracking apparatus, a subject region extraction apparatus, and control methods therefor.
2. Description of the Related Art
An image processing technique of detecting a specific subject in one frame of images that are sequentially supplied in time series, and tracking the detected subject in subsequent frames, is very useful. A technique of detecting and tracking a human body region (for example, a face region) in a moving image, for example, can be employed in a wide variety of fields such as a conference call, a man-machine interface, security, a monitor system for tracking a specific subject, and image compression. Some digital cameras and digital video cameras which incorporate this image processing technique are known to execute exposure control and focus control so as to appropriately capture a subject specified in the image displayed on an electronic viewfinder.
For example, Japanese Patent Laid-Open No. 2005-318554 discloses an image capturing apparatus which detects a face from a captured image, focuses on the face, and captures an image of the face with an exposure optimum for the face. Also, Japanese Patent Laid-Open No. 2001-60269 discloses a tracking process in which a face detected in a certain frame is detected in subsequent frames.
A method which uses a template matching technique, as disclosed in Japanese Patent Laid-Open No. 2001-60269, is known to track a specific subject, detected in a certain frame, in subsequent frames. Template matching is a technique of extracting an image of a subject to be tracked (a template image or a reference image) from a certain frame (to be referred to as a reference frame hereinafter) of a moving image, and searching for a region with a highest degree of correlation with the template image in subsequent frames. The found region is presumed as the position of the template image in each of the subsequent frames, thereby making it possible to track the subject corresponding to the template image.
In a subject tracking method which uses template matching, a subject is tracked based on the degree of correlation between a frame image to be tracked and a reference image (template image). Hence, the subject can be tracked with high accuracy in that method as long as it is captured in the frame image to be tracked in the same way as in the reference image.
However, if, for example, the subject is the face of a person and the face orientation may change or the face may be shielded by a physical object, the tracking accuracy may degrade due to the use of the same reference image. To achieve tracking robust against a change in visual aspect of the subject in the image, it is desirable that the reference image is updated in accordance with the change in visual aspect of the subject.
For example, it is possible to update the reference image so that a region with the highest degree of correlation with the reference image in a frame image to be tracked is used as a new reference image. However, in this method, if the subject is temporarily hidden behind a physical object, a region different from the subject is used as the reference image in the next tracking process, so the subject cannot be precisely tracked in subsequent frames.
Also, it is possible for the above-mentioned reference image to be specified by the user. In this case, the user specifies a region, to be used as a reference image, via an input device which allows him or her to specify an arbitrary region in a frame image, such as a touch panel. With this configuration, the user can easily specify tracking of a specific subject.
Although the reference image desirably contains the subject region to be tracked without any excess and deficiency, the user does not always specify it as such. If a region specified by the user shifts from the intended subject region or is unnecessarily larger than the subject region, the reference image includes images unnecessary for tracking, such as the background, in large amounts. When the subject is tracked using such a reference image which includes unwanted images in large amounts, the subject tracking accuracy may degrade as a region different from the target region may be tracked due to the influence of factors associated with the unwanted images.